1. Technical Field
This disclosure relates to automated software generation, and more particularly, to a system and method for automated software test fixture generation for testing software programs.
2. Description of the Related Art
In modern software development, there are a number of economically driven trends that are changing the way in which software is created. For example, companies that produce software-based products often have “software product lines”, which are families of software that share some over-arching commonalities. These commonalities may occur at a binary level, a source-code level or they may be somewhat more nebulous (e.g., corporate “know-how”). Another emerging trend in software production is that of agile programming, where products are created iteratively, and each successive iteration adds small increments of functionality and resolves problems found in prior iterations. The two practices are in no way exclusive, and many companies use both the “product line” and “agile” concepts to some degree.
Although trends such as software product lines and agile programming can greatly improve programmer productivity in certain settings, they can also cause problems in down-stream portions of the software production process. In particular, they can cause bottle-necks in software testing departments.
For example, consider the practice of iterative programming, which boosts programmer productivity by allowing programmers to successively build on “infrastructure” created in earlier iterations. Unfortunately, iterative programming often reduces the productivity of software testing, since recurring bits of “infrastructure” may change from iteration to iteration and must therefore be tested over and over. In fact, if the testing process is naïve, merely doubling the number of iterations used to produce a piece of software of a given level of complexity can double the work involved in testing. This increase in cost is merely a product of the number of iterations, and the increased expenses do not per se yield benefits in the form of increased software functionality or quality.
Software product lines enhance programmer productivity by promoting code reuse across products. For example, code reuse permits the creation of, say, eight applications with the same effort that it would have taken to write four “no-code-shared” applications, i.e., it has doubled programmer productivity. Unfortunately, this productivity benefit seldom gets passed on to the testing process, as it is difficult or impossible to analytically factor out the eight applications into “four applications worth of testing”. In most cases, the applications are tested as if they were essentially unrelated. The productivity dilemmas introduced by iterative development and those introduced by software product lines are effectively the same. In both cases, there is commonality in structure, but we are unable to analytically determine “what must be tested” and are therefore unable to exploit structural themes in order to reduce testing effort. In the case of iterative programming, the “axis of commonality” is temporal, i.e., iteration N has much in common with iteration N+1. In the case of software product lines, the axis of commonality is across the ensemble of products. Many software companies combine the practices of “iterative development” and “software product lines” to some extent, so their test departments face productivity stress from both the “temporal” and “ensemble” axes.